Image pickup apparatus, image pickup system, signal processing apparatus, and non-transitory computer-readable storage medium

ABSTRACT

An image pickup apparatus includes an image pickup element including a plurality of focus detection pixels configured to generate a pair of image signals based on light beams passing through pupil regions different from each other in an image pickup optical system, a detection unit configured to detect a base line length of the plurality of focus detection pixels and a pupil division direction based on position information of a focus detection region in the image pickup element and information relating to the image pickup optical system, and a calculation unit configured to calculate a phase difference in the pupil division direction by using the pair of image signals.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup apparatus whichperforms focus detection using light beams passing through pupil regionsdifferent from each other in an image pickup optical system.

2. Description of the Related Art

Previously, a focus detection method of using a phase differencedetection method has been known. In the phase difference detectionmethod, a light beam passing through an exit pupil of a lens is divided,and the divided light beams are received by a pair of focus detectionsensors. Then, a drive amount of the lens required to achieve anin-focus state is obtained based on a shift amount of signals outputteddepending on light receiving amounts of the divided light beams, i.e. arelative position shift amount of the light beams in a divisiondirection. Accordingly, since an amount and direction of defocus isobtained when an accumulation is performed once by the focus detectionsensors, a focusing can be performed at high speed.

Japanese Patent Laid-open No. 2000-156823 (paragraphs 0075-0079, andFIGS. 3 and 4, etc.) discloses a configuration in which a sensitiveregion of a light receiving portion is decentered with respect to anoptical axis of an on-chip microlens in apart of light receivingelements (pixels) of an image pickup element to provide a pupil divisionfunction. These pixels are treated as focus detection pixels and arearranged, at predetermined intervals, between image pickup pixels inwhich a sensitive region of alight receiving portion is not decentered,to perform focus detection by the phase difference method. Since theregion where the focus detection pixels are arranged is a defect sectionof the image pickup pixels, image information is interpolated by usinginformation obtained from peripheral image pickup pixels.

Japanese Patent Laid-open No. 2001-305415 (paragraphs 0052-0056, andFIGS. 7 and 8, etc.) discloses a configuration in which a part of lightreceiving portions of pixels in an image pickup element is divided toprovide a pupil division function. It also discloses a configuration inwhich outputs of the divided light receiving portions are independentlyprocessed to perform focus detection by a phase difference method andthe outputs of the divided light receiving portions are added to be usedas an image pickup signal.

Japanese Patent Laid-open No. 2010-140013 discloses a configuration inwhich images are added in an oblique direction considering an influenceof vignetting to be able to perform focus detection for a periphery in ascreen as well.

However, in each of the configurations disclosed in Japanese PatentLaid-open No. 2000-156823, Japanese Patent Laid-open No. 2001-305415,and Japanese Patent Laid-open No. 2010-140013, since the focus detectionis performed by using one-dimensional image, it is weak in an objectimage with a repeated pattern, and an error remains, which is caused byinfluences of an adding direction of generating an image and an obliqueline component included in the object. Therefore, particularly a focusdetection accuracy in the periphery of the screen is deteriorated.

BRIEF SUMMARY OF THE INVENTION

The present invention provides an image pickup apparatus, an imagepickup system, a signal processing apparatus, and a non-transitorycomputer-readable storage medium which are capable of performinghighly-accurate focus detection while avoiding an influence caused by ashape of an object or vignetting.

An image pickup apparatus as one aspect of the present inventionincludes an image pickup element including a plurality of focusdetection pixels configured to generate a pair of image signals based onlight beams passing through pupil regions different from each other inan image pickup optical system, a detection unit configured to detect abase line length of the plurality of focus detection pixels and a pupildivision direction based on position information of a focus detectionregion in the image pickup element and information relating to the imagepickup optical system, and a calculation unit configured to calculate aphase difference in the pupil division direction by using the pair ofimage signals.

An image pickup system as another aspect of the present inventionincludes an image pickup optical system and the image pickup apparatus.

A signal processing apparatus as another aspect of the present inventionincludes an image signal generating unit configured to generate a pairof image signals based on light beams passing through pupil regionsdifferent from each other in an image pickup optical system, a detectionunit configured to detect a base line length of a focus detectionportion and a pupil division direction based on position information ofa focus detection region and information relating to the image pickupoptical system, and a calculation unit configured to calculate a phasedifference in the pupil division direction by using the pair of imagesignals.

A non-transitory computer-readable storage medium as another aspect ofthe present invention stores a program, and the program causes acomputer to execute a process includes generating a pair of imagesignals based on light beams passing through pupil regions differentfrom each other in an image pickup optical system, detecting a base linelength of a focus detection portion and a pupil division direction basedon position information of a focus detection region and informationrelating to the image pickup optical system, and calculating a phasedifference in the pupil division direction by using the pair of imagesignals.

Further features and aspects of the present invention will becomeapparent from the following description of exemplary embodiments withreference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image pickup apparatus in each ofEmbodiment 1 and 2.

FIG. 2 is a cross-sectional view of a pixel of an image pickup elementin the present embodiment (Embodiment 1).

FIG. 3 is a cross-sectional view of pixels of an image pickup element inEmbodiment 2.

FIG. 4 is a pixel array diagram of the image pickup element inEmbodiment 2.

FIGS. 5A and 5B are diagrams of a pupil intensity distribution of theimage pickup element in the present embodiment (Embodiment 1).

FIGS. 6A and 6B are diagrams of illustrating a shape of a vignetting asseen from the image pickup element in each of the embodiments.

FIG. 7 is a diagram of illustrating a relation between the pupilintensity distribution and the shape of the vignetting in each of theembodiments.

FIG. 8 is a diagram of illustrating a relation between a position on theimage pickup element (a screen) and a pupil division state in each ofthe embodiments.

FIG. 9 is a flowchart of illustrating a shooting flow of the imagepickup apparatus in Embodiment 1.

FIG. 10 is a diagram of illustrating a situation where a checked objectexists in a focus detection field of view in each of the embodiment.

FIGS. 11A to 11D are diagrams of illustrating a relation between a pupildivision in an oblique direction and an oblique line in each of theembodiment.

FIGS. 12A and 12B are diagrams of describing a correlation calculationmethod in Embodiment 1.

FIGS. 13A and 13B are diagrams of describing a correlation calculationmethod in Embodiment 2.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described belowwith reference to the accompanied drawings. In each of the drawings, thesame elements will be denoted by the same reference numerals and thedescriptions thereof will be omitted.

First of all, referring to FIG. 2, a structure of a pixel of an imagepickup element in the present embodiment will be described. FIG. 2 is across-sectional view of the pixel of the image pickup element in thepresent embodiment. In FIG. 2, reference numeral 201 denotes amicrolens, reference numeral 202 denotes a color filter, referencenumeral 203 denotes a wiring layer (a semiconductor wiring layer), andreference numerals 204 and 205 denote photoelectric conversion portions.Thus, two photoelectric conversion portions 204 and 205 are provided forone microlens 201, and therefore images (an A image and a B image) inwhich a pupil is divided to the left and right can be obtained. When anoutput of the photoelectric conversion portion 204 is added to an outputof the photoelectric conversion portion 205, an output equivalent to anormal pixel output can be obtained. In the present embodiment, a pixelvalue obtained by adding the outputs of the photoelectric conversionportions 204 and 205 is used to generate an image pickup signal.

Subsequently, referring to FIGS. 5A and 5B, a pupil intensitydistribution of the image pickup element will be described. FIG. 5A is agraph of illustrating a pupil intensity distribution of thephotoelectric conversion portion 204, in which a horizontal axisindicates an incident angle of light with respect to a pixel and avertical axis indicates a sensitivity of the photoelectric conversion.As illustrated in FIG. 5A, with reference to an incident angle of 0degree, a strong intensity is obtained at the right side. FIG. 5B is adiagram of expressing the incident angle in a plane and the sensitivityby a color density. A point in FIG. 5B indicates a center of gravity ofthe sensitivity, and thus the center of gravity of the sensitivity ofthe photoelectric conversion portion 204 is displaced (shifted) to theleft side relative to the center. The center of gravity of thephotoelectric conversion portion 205 (not shown) is displaced to theright side relative to the center. In the present embodiment, a baseline length is determined based on a distance between the two centers ofgravity (the centers of gravity of the photoelectric conversion portions204 and 205).

Subsequently, referring to FIGS. 6A and 6B, a vignetting will bedescribed. FIG. 6A is a diagram of illustrating a position relation of alens as seen from the image pickup element, and FIG. 6B is a diagram ofillustrating a shape of the vignetting. In FIG. 6A, reference numeral601 denotes a plane of the image pickup element, reference numeral 606denotes a rear frame of the lens, and reference numeral 605 denotes afront frame of the lens. As seen from the center of the image pickupelement (a center position of the image pickup element), the front lensof the lens, an aperture stop, and the rear lens of the lens are arrayedas a concentric circle around an optical axis OA, and therefore a lightbeam is blocked by the aperture stop (an aperture stop frame) that hasthe smallest radius. Accordingly, at the center position of the imagepickup element, the shape of the aperture stop influences a lightamount, the eccentricity (displacement) of the center of gravity of thepupil, or the like.

On the other hand, at a position 602 in the vicinity of a diagonalposition of the image pickup element, the light beam is blocked by thefront frame 605 of the lens and the rear frame 606 of the lens inaddition to the aperture stop. Reference numeral 604 denotes avignetting shape that is formed by projecting the front frame 605 of thelens and the rear frame 606 of the lens on an exit pupil (an exit pupilplane 603). In FIG. 6B, the vignetting shape 604 is indicated by ashaded portion. As seen from the position 602 in the vicinity of thediagonal position on the image pickup element, the vignetting shape 604is recognized as a lemon-shaped form.

Subsequently, referring to FIG. 7, a relation between pupil intensitydistributions of the A image and the B image and the vignetting shapewill be described. FIG. 7 is a diagram of illustrating the relationbetween the pupil intensity distributions and the vignetting shape,which illustrates centers of gravity of a region through which light cansubstantially pass in a state where the pupil intensity distributions ofthe A image and the B image are overlapped with the vignetting shape604. In FIG. 7, reference numeral 701 denotes a center of gravity of theA image and reference numeral 702 denotes a center of gravity of the Bimage.

Due to the lemon-shaped vignetting shape 604 formed obliquely, light ofthe A image can be received only at the low side in FIG. 7, and light ofthe B image can be received only at the upper side, and thus the centersof gravity 701 and 702 are decentered (displaced) up and down. In otherwords, although the pupil intensity distribution as a structure of thepixel is pupil-divided to the left and right, it is actuallypupil-divided in an oblique direction by the vignetting shape 604. Thus,since the pupil is divided in the oblique direction, a phase differenceis generated in the oblique direction according to a defocus amount.Consequently, there is a possibility that a correct defocus amountcannot be calculated even when only a phase difference in a horizontaldirection is detected.

Subsequently, referring to FIG. 8, a relation between the position onthe image pickup element (the screen) and the pupil division state willbe described. FIG. 8 is a diagram of illustrating a relation between theposition on the image pickup element (the screen) and the pupil divisionstate. In FIG. 8, the pupil division state is illustrated in which thepupil is divided to the left and right at a position 801 in the vicinityof the center of the optical axis, and therefore a correct defocusamount can be detected if the phase difference in the horizontaldirection is detected correctly.

Although the vignetting shape is a lemon shape at a position 803, thephase difference in the horizontal direction only needs to be detectedsince a pupil division direction is the horizontal direction. However,due to the influence of the vignetting, the base line length at theposition 803 is shorter than that at the position 801. Since the pupilis divided in the oblique direction at a position 802, the phasedifference in the oblique direction needs to be detected. The pupil atthe position 804 is divided in an oblique direction opposite to thedirection for the position 802. Thus, the pupil division direction andthe base line length change depending on a position on the image pickupelement and a state of an exit window of the lens.

Next, referring to FIGS. 11A to 11D, a case in which a pupil is dividedin an oblique direction for a line in the oblique direction as an objectwill be described. FIG. 11A is an example of a case in which an objectof an oblique line exists with respect to the focus detection field ofview. Reference numeral 1101 denotes the focus detection field of view,reference symbol A denotes an object projected as an A image, andreference symbol B denotes an object projected as a B image. Since thebase line length is in the oblique direction, the A image and the Bimage are projected to be shifted from each other by a defocus. Ashifted distance between the A image and the B image (an image shiftamount 1102) is different from an image shift amount 1103 captured inthe focus detection field of view. Thus, in the example illustrated inFIG. 11A, the image shift amount 1103 that is larger than the actualimage shift amount 1102 is detected.

FIG. 11B is an example in which a defocus and an image shift amount 1102are the same as those in FIG. 11A. In FIG. 11B, however, an angle of theoblique line as the object is different from that in FIG. 11A. Theobject illustrated in FIG. 11B is an inclined object in an oppositedirection compared to the object illustrated in FIG. 11A, and thereforethe detected image shift amount 1103 is extremely small. An objectillustrated in FIG. 11C is a V-shaped form. FIG. 11D illustrates animage where the V-shaped object illustrated in FIG. 11C is captured infocus detection fields of view 1104 and 1105. The focus detection fieldof view 1104 indicates the A image and the focus detection field of view1105 indicates the B image. In the state of FIG. 11D, an image shiftamount by which the A image and the B image coincide with each othercannot be detected.

Subsequently, referring to FIG. 10, a problem in a case where pixels areadded in a vertical direction will be described. FIG. 10 is a diagram ofillustrating a situation where a checked object exists in a focusdetection field of view. In FIG. 10, reference numeral 1001 denotes afocus detection field of view. Reference numerals 1007, 1008, 1009, and1010 are image signal waveforms of portions indicated by dashed-twodotted lines 1002, 1003, 1004, and 1005, respectively. Each of the imagesignal waveforms 1007, 1008, 1009, and 1010 contains an edge portion (aluminance edge), and therefore a phase difference between pupil-dividedimages can be correctly detected. Reference numeral 1006 is an imageobtained by adding and averaging the signals in the focus detectionfields of view. Reference numeral 1011 is an image signal waveform ofthe image 1006. The luminance edge which existed before signals in thefocus detection fields of view are added is lost in the image signalwaveform 1011 after the signals are added.

As described with reference to FIGS. 10 and 11, in the presentembodiment, a correlation calculation of a two-dimensional image,instead of a one-dimensional image, is suitable. With respect to thecorrelation calculation of the two-dimensional image, similarly to thecorrelation calculation of the one-dimensional image, a method of usinga sum of absolute differences, called SAD, is in practical use in motionvector detection or an image compression technology. Therefore, it is analgorithm rather suitable for hardware processing than software. Thepresent embodiment detects a phase difference in an oblique direction,and therefore the same method as that of the motion vector detection fora two-dimensional image is used. Hereinafter, specific examples in thepresent embodiment will be described.

EMBODIMENT 1

First of all, referring to FIG. 1, a defocus detection apparatus (animage pickup apparatus) will be described. FIG. 1 is a block diagram ofan image pickup apparatus 100 in the present embodiment. In the imagepickup apparatus 100, reference numeral 101 denotes a lens unit (animage pickup optical system) including a focus lens or an aperture stopmechanism. In the present embodiment, the image pickup apparatus 100 (animage pickup apparatus body) is configured integrally with the lens unit101, but the embodiment is not limited to this. The present embodimentcan also be applied to an image pickup system that is configured by theimage pickup apparatus body and a lens unit removably mounted on theimage pickup apparatus body.

Reference numeral 102 denotes an image pickup element (an image signalgenerating unit) that has a structure of a pixel illustrated for examplein FIG. 2, which photoelectrically converts an object image (an opticalimage) obtained via an image pickup optical system. The image pickupelement 102 is, as described above, includes a plurality of focusdetection pixels that generate a pair of image signals based on lightbeams passing through pupil regions (partial regions in a pupil)different from each other in the lens unit 101 (the image pickup opticalsystem). Reference numeral 103 denotes an A/D converter, which convertsan output signal (an analog signal) of the image pickup element 102 intoa digital signal.

Reference numeral 104 denotes an image signal adding unit (an AB-imageadding unit), which adds image signals (A image signal and B imagesignal) from divided pixels that are output signals of the A/D converter103. An output of the image signal adding unit 104 has a compatibilitywith respect to an output of a typical image pickup element having aBayer array or the like. Reference numeral 105 denotes a signalprocessing unit (an image signal processing circuit), which performspredetermined signal processing on an image signal outputted from theimage signal adding unit 104. Reference numeral 106 denotes a recordingmedium, which records a shot image (an image signal outputted from thesignal processing unit 105).

Reference numeral 107 denotes an image signal separating unit (anAB-image separating unit), which separates and synchronizes the A imagesignal and the B image signal transmitted from the A/D converter 103 bya dot sequential system. Reference numeral 108 denotes a correlationcalculating unit (a calculation unit). The correlation calculating unit108 accumulates the A image signals and the B image signals (the pairsof image signals) outputted from the image signal separating unit 107for one screen, and performs a correlation calculation for a focusdetection region to calculate an image shift amount. In other words, thecorrelation calculating unit 108 calculates a phase difference in apupil division direction (an image shift direction) by using a pair ofimage signals.

Reference numeral 109 denotes a microcomputer (a controller), whichcontrols an entire system of the image pickup apparatus 100. Themicrocomputer 109 outputs an AF field of view position (a focusdetection region) and vignetting information (information relating tothe image pickup optical system) to a pupil division state detectingunit 110. The pupil division state detecting unit 110 detects(calculates) a base line length and a pupil separation angle (a pupildivision direction) based on the information, and outputs the base linelength and the pupil division angle to the microcomputer 109. Themicrocomputer 109 outputs the information relating to the AF field ofview position and shift searching direction to the correlationcalculating unit 108. The correlation calculating unit 108 outputs animage shift amount calculated based on the information to themicrocomputer 109.

Reference numeral 110 denotes a pupil division detecting unit (adetection unit). The pupil division detecting unit 110 detects a baseline length of a plurality of focus detection pixels and a pupildivision direction, based on position information of the focus detectionregion in the image pickup element 102 and information relating to thelens unit 101 (vignetting information).

Next, referring to FIG. 9, a flow of taking an image using the imagepickup apparatus 100 (a method of controlling the image pickupapparatus) of the present embodiment will be described. FIG. 9 is aflowchart of illustrating the flow of taking an image using the imagepickup apparatus 100 (a shooting sequence). Each step of FIG. 9 isperformed mainly by an instruction of the microcomputer 109.

First of all, the image pickup apparatus 100 starts the shootingsequence in Step S900, and then performs exposure and capture for thefocus detection (AF) in Step S901. In this case, the A/D converter 103performs the A/D conversion on a signal exposed by the image pickupelement 102. Then, the A image signal and the B image signal separatedby the image signal separating unit 107 are inputted to the correlationcalculating unit 108. In Step S901, the image signal adding unit 104,the signal processing unit 105, and the recording medium 106 do notoperate.

Subsequently, in Step S902, the microcomputer 109 sets radii of theaperture stop, the front frame, and the rear frame, and distances from asensor (the image pickup element 102) to them, i.e. information relatingto the lens unit 101 (the image pickup optical system). The informationis calculated by the microcomputer 109, or alternatively may be set by auser. Then, the microcomputer 109 transfers, to the pupil division statedetecting unit 110, the radii of the aperture stop, the front frame, andthe rear frame, and the distances from the sensor to them (theinformation relating to the image pickup optical system). In the presentembodiment, the information relating to the image pickup optical systemis information based on a size of the image pickup optical system (theradii of the aperture stop, the front frame and the rear frame of thelens) and the distances from the image pickup element 102 to them (exitwindow information), but the embodiment is not limited to this.

Subsequently, in Step S903, the microcomputer 109 sets a defocus amountdetection area (the focus detection region). Then, the microcomputer 109transfers defocus detection area information (information relating tothe focus detection region) to the correlation calculating unit 108 andthe pupil division state detecting unit 110.

Subsequently, in Step S904, the pupil division state detecting unit 110detects the base line length of the plurality of focus detection pixels(a focus detection portion) and the pupil division direction, based onposition information of the focus detection region in the image pickupelement 102 and the information relating to the image pickup opticalsystem. In the present embodiment, the position information of the focusdetection region is image height information of the focus detectionregion (information relating to a height in a screen, i.e. a height froma center of the image pickup element (an optical axis center)). Thus,the pupil division state detecting unit 110 calculates the base linelength and the pupil division direction by using the exit windowinformation and the pupil intensity distribution.

Specifically, the pupil division state detecting unit 110 calculates thevignetting shape 604 projected on the exit pupil plane 603 by ageometric calculation, based on the position 602 and the informationrelating to the front lens 605 of the lens and the rear lens 606 of thelens (the information relating to the image pickup optical system)illustrated in FIG. 6. Then, the pupil division state detecting unit 110cuts the pupil intensity distribution projected on the exit pupil plane603 by the vignetting shape 604 to obtain a center of gravity of thesensitivity.

When the center of gravity of the sensitivity is obtained based on thepupil intensity distributions of the A image and the B image, the baseline length and the pupil division direction are calculated. In otherwords, the base line length of the plurality of focus detection pixelsis calculated based on a distance between centers of gravity of thesensitivities of the plurality of focus detection pixels determinedbased on the pupil intensity distributions of the plurality of focusdetection pixels and the information relating to the image pickupoptical system. The pupil division direction is a direction which isparallel to a line connecting the centers of gravity of thesensitivities of the plurality of focus detection pixels. The base linelength and the pupil division angle calculated by the pupil divisionstate detecting unit 110 are outputted to the microcomputer 109.

Subsequently, in Step S905, the microcomputer 109 specifies a phaseshift direction (the pupil division direction), and the correlationcalculating unit 108 detects (calculates) the phase difference in thepupil division direction by using the pair of image signals. In thiscase, the microcomputer 109 sets the pupil division direction (the pupildivision angle) to the correlation calculating unit 108 so as toinstruct the correlation calculating unit 108 to perform a search in thepupil division direction. The correlation calculating unit 108 performsthe correlation calculation by using the pupil division angle set by themicrocomputer 109.

Referring to FIGS. 12A and 12B, the correlation calculation using thepupil division direction (the pupil division angle) will be described.FIGS. 12A and 12B are diagrams of describing a correlation calculatingmethod in the present embodiment. In FIG. 12A, “A” is a V-shaped objectwhich is projected on an A image field of view (an A image pixel), and“B” is an object, which is the same as the object A, projected on a Bimage field of view (a B image pixel). Reference numeral 1101 denotesthe A image field of view. FIG. 12B is a diagram of an extracted imagewhich is projected on the A image field of view 1101.

Conventionally, the A image field of view was used also for the B image.In the correlation calculation of the present embodiment, however, thecorrelation calculation is performed sequentially for the field of viewindicated by dotted lines in FIG. 12A while searching a portioncorresponding to the A image field of view 1101, and a degree ofcoincidence is evaluated. In the present embodiment, the B image fieldof view is shifted in an oblique direction with respect to the A imagefield of view, and its direction is the pupil division direction or adirection which is determined depending on the pupil division direction.An arrow 1102 indicates a direction and a distance in which the image isactually shifted according to the pupil division. In the presentembodiment, a location where the degree of coincidence of the pair ofimage signals is maximized is searched along the direction of the arrow1102 (the pupil division direction). Reference numeral 1201 is the Bimage field of view where the degree of coincidence with the A imagefield of view 1101 is maximized.

FIG. 12B illustrates an image projected on each of the A image field ofview 1101 and the B image field of view 1201 where the degree ofcoincidence with the A image field of view 1101 is highest. Obtainingthe phase difference by a subpixel unit using correlation amounts beforeand after reaching the location where the degree of coincidence ishighest is similar to a well-known phase difference method. Thecorrelation calculating unit 108 multiplies a coefficient determineddepending on the base line length by the phase difference obtained asdescribed above, and thus it can calculate a defocus amount (an imageshift amount).

Subsequently, in Step S906 of FIG. 9, the microcomputer 109 determineswhether or not the phase difference detection for the entire focusdetection area (the focus detection region) is completed. When the phasedifference detection for the entire focus detection area is notcompleted, the flow returns to Step S903, and Steps S903 to S906 arerepeated. On the other hand, when it is determined that the phasedifference detection for the entire focus detection area is completed inStep S906, i.e. when the image shift amount is obtained, the flowproceeds to Step S907.

In Step S907, the microcomputer 109 detects a focus position for theobject (main object) and moves the lens unit 101 to perform a focuscontrol. In other words, the microcomputer 109 (the controller) performsthe focus control so that the image shift amount calculated by thecorrelation calculating unit 108 decreases. Subsequently, in Step S908,the microcomputer 109 controls the image signal adding unit 104, thesignal processing unit 105, the recording medium 106, and the like, toshoot an image.

As a result, in the present embodiment, the influence of the vignetting,the base line length, and the pupil division direction are calculatedbased on an optical condition. Then, a phase shift detection directionis determined according to the calculated influence of the vignetting,the base line length, and the pupil division direction to perform thephase difference detection.

Embodiment 2

Next, a defocus detection apparatus (an image pickup apparatus) inEmbodiment 2 of the present invention will be described. First of all,referring to FIG. 3, a structure of pixels of an image pickup element inthe present embodiment will be described. FIG. 3 is a cross-sectionalview of the pixels of the image pickup element in the presentembodiment. Reference numeral 201 denotes a microlens, reference numeral202 denotes a color filter, and reference numeral 203 denotes a wiringlayer. Reference numeral 301 denotes a pixel (an image pickup pixel)which is used to generate a normal image signal. Reference numeral 303denotes a pixel (a focus detection pixel) which is used to generate afocus detection signal. Reference numeral 302 denotes a light blockingportion which extends the wiring layer 203 to an upper portion of thepixel to block the light. With respect to the pixel 303 illustrated inFIG. 3, light enters the pixel 303 only through an opening portion 305at the right side, and therefore the pixel 303 is in a state where itdoes not have sensitivity at the left side in the pixel 303, which is ina state where a pupil is decentered (displaced).

FIG. 4 is a pixel array diagram as seen from a front side of the imagepickup element in the present embodiment. Reference numeral 301 denotesa normal pixel (an image pickup pixel), and reference numerals 303 and304 denote pixels (focus detection pixels) that generate focus detectionsignals. The image pickup element of the present embodiment is differentfrom that of Embodiment 1 in that the pupil-divided pixels (the focusdetection pixels) are arranged discretely. In the present embodiment,similarly to Embodiment 1, there is an influence caused by thevignetting, and therefore the pupil division state changes depending onan image height or a position or a size of the lens.

Subsequently, referring to FIGS. 13A and 13B, a correlation calculationin the present embodiment will be described. FIGS. 13A and 13B arediagrams of describing a correlation calculation method in the presentembodiment. FIG. 13A illustrates focus detection pixels arrangeddiscretely and a V-shaped object which is projected on the A image pixeland the B image pixel. Reference numeral 1301 denotes a field of view ofthe A image pixel. The field of view (an image) in the presentembodiment has two dimensions (a two-dimensional image). In the presentembodiment, however, since the focus detection pixels are arrangeddiscretely, each of the A image and the B image actually obtained is asmall image like a reduced image with respect to an image pickup image.

FIG. 13B illustrates a searching direction in the present embodiment. Inthe present embodiment, when searching a location where a degree ofcoincidence of the B image field of view is high with respect to the Aimage field of view, a combination of constant widths in a longitudinaldirection and in a lateral direction is a search range, instead of usinga pupil division direction previously calculated. In other words, acorrelation calculating unit 108 (a calculation unit) of the presentembodiment searches a location where a degree of coincidence of a pairof image signals is maximized along a first direction and a seconddirection different from the first direction to perform atwo-dimensional correlation calculation, and thus it calculates a phasedifference and a direction of the phase difference.

By using this searching method, a circuit which is common to a motionvector detecting circuit can be used in a manner of time division.Therefore, the scale of circuits can be reduced. In addition, forexample, since the searching can start before the calculation result ofStep S904 in FIG. 9 is obtained, a calculation time can be reduced byperforming a parallel calculation.

Preferably, the microcomputer 109 (a determination unit) performs thetwo-dimensional correlation calculation to detect the phase difference,and then determines whether or not the direction of the phase differencecoincides with the pupil division direction. In the present embodiment,the two-dimensional correlation calculation is performed to be able todecrease a probability of error detection compared to a one-dimensionalcorrelation calculation, and it is more preferred that the generation ofthe error detection is detected by using the pupil division state (thepupil division direction). Thus, in the present embodiment, the phasedifference detected by using the two-dimensional correlation calculationis compared to the detection result of the pupil division state toperform an error determination.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

For example, although the SAD (Sum of Absolute Difference) is used asthe correlation calculation method in each embodiment, each embodimentis not limited to this and a similar effect can be obtained even when adifferent kind of correlation values capable of detecting the phasedifference is used. Although the method in each embodiment is used as apart of a so-called autofocus function in which a focusing lens isdriven to be focused on the object as a result of the defocus detection,it may also be used for obtaining a distance map and as a result it maybe applicable for a photometry, a colorimetry, or the like for eachobtained distance. In addition, in each embodiment, although thecorrelation calculation method is performed by the image pickupapparatus, each embodiment is not limited to this and the method mayalso be applicable for a technology of making a distance map by using apost-processing apparatus (a signal processing apparatus) such as apersonal computer which processes shot image data. For example, themicrocomputer 109 (the controller) adds, to an image, informationrelating to a depth direction of an object based on an image shiftamount.

In each embodiment, the image pickup element is used as an image signalgenerating unit which generates a pair of image signals to perform thepupil division, but each embodiment is not limited to this. For example,the same effect as that of each embodiment can be obtained even in asystem of performing spectrometry and pupil division using a relay lensfor a light beam from one image pickup lens. In each embodiment, thepupil division state (the pupil division direction) is detected by thegeometric calculation, and the similar effect can be obtained even whenusing a previously calculated table corresponding to a control of alens.

Other Embodiments

Embodiments of the present invention can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiment (s) of the present invention, andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiments. The computer may comprise one or more of acentral processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

According to each embodiment described above, an image pickup apparatus,an image pickup system, a signal processing apparatus, and anon-transitory computer-readable storage medium which are capable ofperforming highly-accurate focus detection while avoiding an influencecaused by a shape of an object or vignetting can be provided.

This application claims the benefit of Japanese Patent Application No.2013-047442, filed on Mar. 11, 2013, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image pickup apparatus comprising: an imagepickup element including a plurality of focus detection pixelsconfigured to generate a pair of image signals based on light beamspassing through pupil regions different from each other in an imagepickup optical system; a detection unit configured to detect a base linelength of the plurality of focus detection pixels and a pupil divisiondirection based on position information of a focus detection region inthe image pickup element and information relating to the image pickupoptical system; and a calculation unit configured to calculate a phasedifference in the pupil division direction by using the pair of imagesignals.
 2. The image pickup apparatus according to claim 1, wherein:the position information of the focus detection region includes imageheight information of the focus detection region, and the informationrelating to the image pickup optical system includes exit windowinformation based on a size of the image pickup optical system and adistance from the image pickup element.
 3. The image pickup apparatusaccording to claim 1, wherein the base line length of the plurality offocus detection pixels is calculated based on a distance of centers ofgravity of sensitivities of the plurality of focus detection pixelsdetermined based on a pupil intensity distribution in the plurality offocus detection pixels and the information relating to the image pickupoptical system.
 4. The image pickup apparatus according to claim 3,wherein the pupil division direction is a direction that is parallel toa line connecting the centers of gravity of the sensitivities of theplurality of focus detection pixels.
 5. The image pickup apparatusaccording to claim 1, wherein the calculation unit multiplies acoefficient determined depending on the base line length by the phasedifference to calculate an image shift amount.
 6. The image pickupapparatus according to claim 1, further comprising a controllerconfigured to perform a focus control so that the image shift amountcalculated by the calculation unit decreases.
 7. The image pickupapparatus according to claim 1, wherein the calculation unit searches alocation where a degree of coincidence of the pair of image signals ismaximized along the pupil division direction to calculate the phasedifference.
 8. The image pickup apparatus according to claim 1, whereinthe calculation unit searches a location where a degree of coincidenceof the pair of image signals is maximized along a first direction and asecond direction different from the first direction to perform atwo-dimensional phase calculation to calculate the phase difference anda direction of the phase difference.
 9. The image pickup apparatusaccording to claim 8, further comprising a determination unit configuredto determine whether or not the direction of the phase differencecoincides with the pupil division direction.
 10. An image pickup systemcomprising: an image pickup optical system; an image pickup elementincluding a plurality of focus detection pixels configured to generate apair of image signals based on light beams passing through pupil regionsdifferent from each other in the image pickup optical system; adetection unit configured to detect a base line length of the pluralityof focus detection pixels and a pupil division direction on positioninformation of a focus detection region in the image pickup element andinformation relating to the image pickup optical system; and acalculation unit configured to calculate a phase difference in the pupildivision direction by using the pair of image signals.
 11. A signalprocessing apparatus comprising: an image signal generating unitconfigured to generate a pair of image signals based on light beamspassing through pupil regions different from each other in an imagepickup optical system; a detection unit configured to detect a base linelength of a focus detection portion and a pupil division direction basedon position information of a focus detection region and informationrelating to the image pickup optical system; and a calculation unitconfigured to calculate a phase difference in the pupil divisiondirection by using the pair of image signals.
 12. The signal processingapparatus according to claim 11, wherein the calculation unit multipliesa coefficient determined depending on the base line length by the phasedifference to calculate an image shift amount.
 13. The signal processingapparatus according to claim 12, further comprising a controllerconfigured to add, to an image, information relating to a depthdirection of an object based on the image shift amount.
 14. Anon-transitory computer-readable storage medium storing a program, theprogram causes a computer to execute a process comprising: generating apair of image signals based on light beams passing through pupil regionsdifferent from each other in an image pickup optical system; detecting abase line length of a focus detection portion and a pupil divisiondirection based on position information of a focus detection region andinformation relating to the image pickup optical system; and calculatinga phase difference in the pupil division direction by using the pair ofimage signals.